1. Technical Field
The present invention is generally directed to an improved data processing system. More specifically, the present invention is directed to an apparatus and method for providing remote access redirect capability in a channel adapter of a system area network.
2. Description of Related Art
In a System Area Network (SAN), such as an InfiniBand (IB) network, the hardware provides a message passing mechanism that can be used for Input/Output devices (I/O) and interprocess communications (IPC) between general computing nodes. Processes executing on devices access SAN message passing hardware by posting send/receive messages to send/receive work queues on a SAN channel adapter (CA). These processes also are referred to as “consumers.”
The send/receive work queues (WQ) are assigned to a consumer as a queue pair (QP). The messages can be sent over five different transport types: Reliable Connected (RC), Reliable Datagram (RD), Unreliable Connected (UC), Unreliable Datagram (UD), and Raw Datagram (RawD). Consumers retrieve the results of these messages from a completion queue (CQ) through SAN send and receive work completion (WC) queues. The source channel adapter takes care of segmenting outbound messages and sending them to the destination. The destination channel adapter takes care of reassembling inbound messages and placing them in the memory space designated by the destination's consumer.
Two channel adapter types are present in nodes of the SAN fabric, a host channel adapter (HCA) and a target channel adapter (TCA). The host channel adapter is used by general purpose computing nodes to access the SAN fabric. Consumers use SAN verbs to access host channel adapter functions. The software that interprets verbs and directly accesses the channel adapter is known as the channel interface (CI).
Target channel adapters (TCA) are used by nodes that are the subject of messages sent from host channel adapters. The target channel adapters serve a similar function as that of the host channel adapters in providing the target node an access point to the SAN fabric.
Thus, with the SAN architecture described above, an Ethernet device driver can communicate with an Ethernet adapter by posting send/receive messages to a Host Channel Adapter (HCA) and retrieve the results of these messages through the HCA's Send and Receive Work Queues. The Ethernet adapter includes a Target Channel Adapter, which is the component that attaches to the SAN. Thus, to attach to a Local Area Network (LAN), such as an Internet Protocol (IP) and Ethernet network, an Ethernet adapter is needed as well as a switch or router that attaches the Ethernet adapter to the IP based LAN.
In a SAN architecture, such as InfiniBand, reads and writes from/to an application's memory space are performed directly via the host channel adapters. Because of this, when data integrity operations must be performed on the application's memory space, there is a possibility that a read and/or write request pending in a host channel adapter may attempt to access a portion of the application's memory space that is being operated on by the data integrity operations. This may result in a loss of data integrity. That is, the actual state of the data in memory may not be as expected by the system. This is often referred to as a race condition and may lead to systemic problems or faults.
For example, assume that a portion of memory is being written to disk as part of a data integrity operation. If a host channel adapter has a pending write operation to this portion of memory, the write operation to this portion of memory may be performed simultaneously with the write operation of this portion of memory to disk. As a result, there is a mismatch between the data in the memory and the data in the copy of this portion of memory that is written to disk. Due to this mismatch, if a failure occurs requiring a rollback of the application's state, the correct state of the application may not be achievable since the copy of the portion of memory on disk does not include the write to the memory that occurred at the same time as the write to the disk nor does the operation log have the write operation listed after the time point of the write to disk. As a result, the write operation to the memory will not be replayed after rollback.
Therefore, it would be beneficial to have a system and method for ensuring the integrity of data during data integrity operations in a system area network.